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special treatment for images lying on or near borders and 
pre-processing of test images. 
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SPATIAL STANDARD OBSERVER 

The invention described herein was made by employees of 
the United States Government and may be manufactured and 
used by or for the Government for governmental purposes 
without payment of any royalties thereon or therefor. 

BACKGROUND OF INVENTION 

1. Technical Field of the Invention 

This invention relates generally to the field of devices and 
methods for the specification and measurement of the percep- 
tual intensity of one or more visual images and, more particu- 
larly, to the rapid and efficient determination of a visibility 
metric for such images. 

2. Description of the Prior Art 

Vision is the means by which most people acquire and 
process information about the world around them. Numerous 
ob jects intended for human use include a component of infor- 
mation to be identified visually by a human observer. Some 
everyday examples include information displayed on a screen 
or page, keys or buttons to be pressed on a keyboard, tele- 
phone, calculator, remote control unit, among many other 
examples. Therefore, it is reasonable that the design of such 
objects include specifications to insure that the visual infor- 
mation is accessible to typical human observers, that is. that 
the information is visible. Providing a means for measuring 
and specifying visibility, a “visibility metric," is an objective 
of the present invention. 

A significant challenge in designing standards for visibility 
is that such standards are based upon models of the human 
visual sense. However, vision is a complex and only partially 
understood process. Previous standards for visibility have 
thus tended to be complex, difficult to use and not sufficiently 
general to serve as a standard method or methods for the 
specification and measurement of visibility. The performance 
of various visibility metrics has been reviewed by Ahumada 
and coworkers in two publications: Society for In formation 
Display, International Symposium, Digest of Technical 
Papers Vol. 24. pp. 305-308 (1993) and Vol. 26. pp. 45-48 
(1995), the contents of both publications are incorporated 
herein by reference. 

Other examples of visibility metrics include the work of 
Lubin and co-workers U.S. Pat. No. 6,654.504. US Patent 
Application Publication 2002/0031277 and “A Human Sys- 
tem Model for Objective Picture Quality Measurements," 
Proceedings, International Broadcasters \ Convention , 

Amsterdam. The Netherlands, pp. 498-503 (1997). These 
methods developed by Lubin and co-workers require exten- 
sive calibration for each application in addition to suffering 
from the disadvantage of complexity. These methods are 
chiefly intended for image quality evaluation. 

Other methods for estimating visibility include those of 
Barten, “The SQRI Method: A New Method for the Evalua- 
tion of Visible Resolution on a Display," Proceedings of the 
Society for Information Display, Vol. 28, pp. 253-262 ( 1 987). 
In addition to complexity, the Barten method suffers from the 
further disadvantage of being appropriate primarily for the 
specification of displays such as television monitors. 

Standards for the measurement and specification of color 
are known in the art and widely used. However, such color 
standards typically do not address the spatial pattern 
employed in a visual signal (for example, the shape of a 
letter). Consequently, such methods are not appropriate for 
specifying or measuring visibility. ■ 

Thus, a need exists in the art for a standard specification 
and measurement of visibility, sufficiently general to be 
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applicable to large classes of visual information but suffi- 
ciently simple for widespread implementation and use, 
including embedding into inexpensive systems. 

SUMMARY OF THE INVENTION 

Accordingly and advantageously, the present invention 
relates to systems and techniques for processing visual infor- 
mation to produce a single numerical value for the visibility 
metric indicative of a Spatial Standard Observer (SSO). 
Advantages of the SSO include a simple and efficient design 
that produces an accurate visibility metric with a relatively 
few calculations. 

Some embodiments of the present invention use a 
Minkowski sum directly over filtered image pixels. This tech- 
nique avoids the need for complicated spatial frequency filter 
banks, with a corresponding gain in simplicity and computa- 
tional efficiency. 

A particular form of Contrast Sensitivity Filter (CSF) is 
used in some embodiments of the present invention which 
combines radial- and oblique-effect filters. This permits accu- 
rate visibility predictions of the visibility of oblique patterns 
such as half-toning and rasterizing artifacts. 

Viewing distance and image resolution are jointly treated 
in an advantageous manner in some embodiments of the 
present invention. The use of this feature causes the computed 
value of image visibility to be substantially independent of 
image resolution (except to the extent that the resolution 
actually alters the visibility of the information in the image). 

A window function is advantageously employed in some 
embodiments of the present invention in such a manner as to 
represent the reduction in visibility with distance from the 
observer’s region of fixation. 

It is advantageous in some embodiments of the present 
invention to use convolution operations along with the win- 
dow function. In this manner it is feasible to simulate the 
scanning of an image by the eye of the observer. 

Pooling the data accumulates the visibility over the scan 
and is advantageously employed in some embodiments of the 
present invention. 

When images are located near a border region, it may occur 
that the border has a markedly different intensity (typically 
darker) than that of the image and the general image back- 
ground. In such cases, it is advantageous in some embodi- 
ments of the present invention to introduce special procedures 
for handling border effects. Two examples are presented. One 
includes at least a portion of the border into the definition of 
“image" leading to a enhanced image that is then processed 
by the SSO. Another approach is to attenuate the image con- 
trast near the border. 

The SSO provides a standardized measure of visibility, 
allowing comparisons to be made of visibility measurements 
taken in a wide variety of applications, locations and times. 
Manufacturing and engineering specifications of visibility in 
standardized units can then be made. 

Furthermore, SSO visibility measurements are not limited 
bv target size. Thus, very large or very small displays can use 
SSO. 

The SSO further provides the feasibility of making simple, 
automated measurements of the visibility of visual informa- 
tion, not requiring the use of human observers to estimate 
visibility. Simplicity of measurement is an important feature 
of SSO in order to allow SSO to be adopted in a wide variety 
of applications and at low cost. 

SSO has numerous potential areas of application. We note 
a few applications as illustrative of the utility of SSO, not 
thereby limiting the scope of SSO to only those enumerated. 
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Many other applications are apparent to those with ordinary 
skill in the art, within the scope of the present invention. 
Possible applications include: 

Photometric Instruments incorporating SSO to produce a 
“spatial photometer” for the measurement of the visibil- 
ity of spatial patterns. 

Imaging Devices and Systems employing SSO to calculate 
the visibility of targets as viewed through those systems 
such as infrared viewing systems and remote viewing 
systems (e.g., as in unmanned aerial vehicles). 

Copier Manufacturing employing SSO to measure the vis- 
ibility of defects produced by copiers and thus test the 
copier and/or improve copier design. 

Video Codecs employing SSO in testing and/or design to 
measure the visibility of image compression artifacts 
with a view towards reducing visible defects and 
increasing bitrale. 

Display Manufacturing employing SSO to detect and mea- 
sure visible artifacts w ith a view' towards improving and 
automating product quality control and output by only 
rejecting devices having visible artifacts. 

Graphics Software employing SSO to estimate the visibil- 
ity of graphic elements and/or to estimate the visibility 
of artifacts due to the rendering process. 

Predicting Visual Performance of Humans Following 
Vision Correction using SSO and thereby pre-evaluate 
the relative efficacy of various correction procedures 
before surgery. 

Digital Watermarking employing SSO to calculate the vis- 
ibility of a recoverable signature labeling an image that 
is intended to be invisible to a human viewer. 

These are among the advantages achieved in accordance 
with various embodiments of the present invention as 
described in detail below. 

BRIEF DESCRIPTION OF THE DRAWINGS 

To facilitate understanding, identical reference numerals 
have been used, where possible, to designate identical ele- 
ments that are common to the figures. 

The techniques of the present invention can readily be 
understood by considering the following detailed description 
in conjunction with the following drawings, in which: 

FIG. 1 depicts a high-level block diagram of a typical 
embodiment of a Spatial Standard Observer used to compute 
a visibility metric. 

FIG. 2 depicts a typical target adjacent to a dark border 
surrounding the image. 

FIG. 3 depicts a typical border aperture function that, in 
this example, has 240 columns, 180 rows, each pixel is ( , /6o) 
degree in height and width. The value of the parameter bscale 
is 0.50 deg., and bgain is 1 . 

FIG. 4 depicts a high-level block diagram of an exemplary 
computer system that can be used for implementation of 
techniques of the Spatial Standard Observer. 

DETAILED DESCRIPTION OF THE INVENTION 

After considering the following description, those skilled 
in the art will clearly realize that the teachings of the invention 
can be readily utilized for determining the probable visibility 
of various graphical or visual depictions and displays as 
viewed by a typical human observer. In particular, the present 
invention relates generally to systems and techniques for 
processing one or more images to produce a single numerical 
value, or “visibility metric,” indicative of a “Spatial Standard 
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Observer” (SSO). Advantages of the present invention 
include techniques for the rapid evaluation of the SSO. 

The present invention relates generally to devices and 
methods for the measurement and/or for the specification of 
the perceptual intensity of a visual image. Other embodi- 
ments relate generally to devices and methods for the mea- 
surement and/or for the specification of differences in per- 
ception or “perceptual distance” between two or more visual 
images. Such devices and methods can be advantageously 
used in situations in which it is desired to measure or to 
specify visibility or visual intensity. Examples include the 
determination of visibility and/or discriminability of text, 
graphic elements, labels, icons, among other visual images. 
Examples also include the determination of visibility and/or 
discriminability between images, such as an original image 
and a compressed digital form of that image. Some embodi- 
ments of the present invention can also be advantageously 
used to quantify the visibility of blemishes on a display as 
might be useful, for example, in providing objective determi- 
nations of pass/fail criteria in the manufacture of displays. 

In essence, various embodiments of the present invention 
operate on a digital image (or an analog image following 
digitization) or on a pair of digital images. An arbitrary num- 
ber of images can be compared by repeated pairwise com- 
parisons. Thus, for economy of language we will describe 
applications of the present invention to a single digital image 
or to the comparison of two digital images, understanding that 
this is by way of illustration and not limitation since multiple 
images can be handled by multiple applications of such pair- 
wise comparisons. Analogue images can be handled within 
the scope of the present invention following digitization by 
any of numerous digitization techniques well-known in the 
art, such as use of a digital camera, a scanner, among other 
devices and digitization techniques known in the field. 

In the comparison of two digital images, it is advantageous 
in some embodiments of the present invention to pre-process 
the images to erase any inessential difference before present- 
ing them as input to the SSO. Such pre-processing removal of 
inessential differences can improve the speed to SSO process- 
ing, further enhancing the range of potential applications 
amenable to SSO processing. 

Also, by way of illustration and not limitation, it will be 
presumed in our descriptions that the images are viewed on a 
particular display called the reference display, and viewed at 
a particular viewing distance. Techniques are well-known in 
the art for translating an image on a non-reference display into 
a digital representation as it would appear on the reference 
display, and for translating from an arbitrary viewing distance 
and angle to a standard viewing distance and angle. 

Typical inputs in the construction of a Spatial Standard 
Observer (SSO) are two digital images having (or scaled so as 
to have) the same size, called herein a test image and a 
reference image. G(x,y) is defined to be the grayscale of the 
pixel at column x and row y; G /<?5 ,(x,y), O reference (\,y) for the 
test and reference images respectively. We take the dimension 
of the image to be n v pixels in the x direction (width) and n v 
pixels in the y direction (height). Typical values are n v =640 
and n v =480. 

Letting s v and s v be the viewing angles subtended by the 
image in the x and y directions respectively, the viewing 
angles s A , s v can be derived from the viewing distance and the 
image size in the plane of the display by the use of Eq. 1 twice, 
once to compute s v and once to compute s v . 
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tan{( n * size( degrees) / 360) = Eq. 1 a 

(0.5 * size(cm))/ viewing distance (cm) 

360 size(cm) Eq. lb 

size( degrees) = — : — : — — - 

2 n viewing distance (cm) 


Eq. 1 b follows from Eq. 1 a only when the ratio (size/( viewing 
distance)) is much less than one. But this is true in virtually all 
cases of practical interest so we use Eq. lb hereinafter. Also, 
the designation of cm in Eq. 1 a and 1 b is for convenience, 
since it is only necessary that “size” and “viewing distance” 
be expressed in the same units of length. 

The width and height of each pixel, p v and p v respectively, 
are given by Eq. 2 with p v , p in degrees if s v and s v are in 
degrees. Typical values are s x =8 deg. and s v =6 deg. yielding 
typical values for p v =p v =( Vso) deg. 


The test and reference images. G„, v/ (x,y) and G reference (x*y) 
respectively, may contain noise, or may differ in those image 
components having high spatial frequencies whose visibili- 
ties are not of interest for the particular image analysis under 
consideration. In addition, the images may be captured at a 
higher resolution or larger area than is necessary for the 
particular image analysis. For these and other reasons, it may 
be useful to pre-process the test and reference images to 
remove noise, remove high frequency components and other 
components not significantly affecting the visibility analysis, 
to reduce image resolution, and/or to crop the image to a 
rectangle of interest (or other convenient shape). Such opera- 
tions can be performed by filtering, downsampling and crop- 
ping, pursuant to some embodiments of the present invention. 
Such operations are optional and, when employed, can be 
employed in any combination, sequence or number. That is, 
multiple steps of each operation can be performed whenever 
advantageous to do so, and the sequence of various operations 
or combinations can also be adjusted for the particular image 
processing task at hand. To be concrete in our description, we 
describe typical pre-processing operations, individually and 
in a particular sequence, understanding thereby that the 
present invention is not limited to the particular steps, 
sequence, number or type of operations described. 

It is convenient in some embodiments to pre-filter the test 
and reference images by convolution with a pre-filter function 
PF(x,y) pursuant to Eq. 2. 1 

G'(x,y)=PF(x,y)(x)G(x,y) Eq. 2. 1 

forG^ v/ (x, y ) and G reference {xy) respectively. The G' function 
of Eq. 2. 1 , the pre-processed image, is then used in place of G 
in subsequent image processing, including in Eqs. 3. 4 and 
following. 

In some embodiments of the present invention, it is conve- 
nient to use a pre-filter function PF(x,y) given by Eq. 2.2. 


PF(x, y) = PF(r) Eq. 2.2 

= tE xp(-x( — ) ) 

pscale~ ' Upscale' ’ 


-continued 

r- yj(xpx) 2 +(V/J v ) : 


in which pscale is a parameter, conveniently taken to be 0. 1 25 
degree in some embodiments. 

The test and reference images can be downsampled by 
integer factors in the x and y directions {d v . d v } respectively, 
by selecting every d x -th column and d v -th row from the origi- 
nal image to create a new, downsampled image G"(x,y). This 
operation is conveniently expressed in terms of a “downsam- 
pling operator” DS as 

G" (aw )=DS( G'(x,y).6 x M % .) Eq. 2.3 

The new dimensions of the test and reference images in the 
x and y directions are thus given as n x ' and n v ’ as in Eq. 2.4. 



in which the function “Floor| ]” returns the nearest integer less 
than or equal to its argument. Typical values for d v and d v are 
d x =d v =4. 

Eq. 2.3 uses the pre-processed image G' from Eq. 2. 1 as the 
image from which the downsampled image G" is derived. 
This is a particular example presented to be concrete in our 
description and not intending to limit the scope of the present 
invention. Although downsampling is almost always pre- 
ceded by filtering to avoid aliasing, downsampling can be 
performed on an image with or without pre-filtering. 

The image G. G’ or G" can be cropped to a rectangle of 
interest ROl. For definiteness, we describe cropping the G" 
image having dimensions n v ' and n v '. It is convenient to 
describe the ROI by the pixel coordinates of its lower left 
comer {x 7J , y 7 7 } and upper right comer {x 7/7? , y f/y? } respec- 
tively. Cropping is conveniently performed by deleting from 
the image rows 1 through (y 77 -l ) inclusive, and rows (y 7/A ,+ 
1 ) through n ' inclusive, as well as columns 1 through (x 77 -l ) 
inclusive, and columns (x 7/77 +l) through n v ' inclusive. The 
dimensions of the new, cropped image are thus 

n x"- x UR~ X LL + l 

n y"=yuR-yu+\ Eq. 2.5 

If the pre-processing techniques are used, singly or in 
combination, the resulting output images (test and reference) 
are considered the input images to the other image processing 
procedures described herein. New image dimensions (if 
present) should also be used. 

If a reference image is not readily available, it is convenient 
in some embodiments of the present invention to create one 
by deleting the target or structural component from a copy of 
the test image. If the target is confined to a local region on an 
otherwise uniform image with graylevel G () , then it is conve- 
nient in some embodiments of the present invention to create 
a reference image as a uniform image having the same size as 
the test image with a graylevel also equal to G () . Typical 
images are depicted as 100 in FIG. 1 with test image 100a and 
reference image 100/?. The structural component of the test 
image 100a is shown adjacent to the image field merely for 
clarity of depiction, understanding that the images are actu- 
ally superimposed. 
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If a reference image is not available, some embodiments of 
the present invention obtain a reference image by processing 
the test image, for example, convolving the test image with a 
reference filter, RF(x,y). It is advantageous in some embodi- 
ments to pre-process the test image pursuant to one or more of 
the pre-processing techniques described herein (or others 
known in the held) before application of the reference filter, 
that is, convolve RF with G, G’, G" or equivalents, among 
others. 10 

In some embodiments, it is convenient to create a reference 
image by smoothing the test image and thereby suppress from 
the test image the signals whose visibility is of interest. For 
example, smoothing can conveniently be carried out with a 
Gaussian reference filter having the form given by Eq. 2.6. 

RF(x. y) = RF(r) Eq. 2.6 



r = yj (xp x ) 2 + ( y /» v ) 2 


“rscale” is a parameter conveniently chosen to be 2 degree. 

The reference image is then created by convolving the test 
image with the reference (liter, Eq. 2.6, either by employing 
conventional convolution (e.g., Eq. 5a, 5b, 5c) or, advanta- 30 
geously according to some embodiments of the present inven- 
tion, using “confined convolution,” denoted by a “eonlined 
convolution operator” (x) r , as applied in Eq. 2.7. 

G'"(x.y)=RF(x.y)(x) c G"(x,y) Eq. 2.7 35 

Eq. 2.7 depicts the example in which the pre-processed 
image G"" is convolved by confined convolution to produce a 
reference image G"’, understanding that pre-processing the 
test image is optional and conventional or other forms of 40 
convolution can be employed. 

Confined convolution offers some advantages in image 
processing. In standard cyclic convolution, the edges of the 
image are considered to be connected. Thus, image content 45 
close to one edge of the image may be spread over to the 
opposite edge, which is sometimes called the “wrap-around 
problem.” Confined convolution is a form of convolution 
which avoids the wrap-around problem by, in effect, discon- 
necting the opposing edges of the image. 50 

Confined convolution makes use of a “Pad-Convolve- 
Crop” (PCC) operator. The operands of the PCC operator are 
a general image function, l(x,y), and a kernel K(x,y) in which 
the kernel has k v columns and k v rows. The image I(x,y) is 55 
augmented or “padded” with rows and columns containing 
entries having a value of 0, such that the padded image has k v 
additional columns (of all 0's) and k v additional rows (of all 
0’s) in comparison with I(x,y). This padded l(x,y) is con- 
volved with the kernel K(x,y). The image resulting from this 60 
convolution is then restored to the original image size by 
removing the added k rows and k v columns. This sequence of 
operations defines the PCC operator operating on K and I, 
denoted as PCC(K(x,y),I(x,y)). 65 

The confined convolution of K(x,y) with I(x,y) is then 
given by Eq. 2.8. 


R(x. v) ®c /(a, y) = 


PCC(K{ a. y>. /U. y» 




Eq. 2.8 


in which l(x,y) is an image (array ) all of whose entries=l and 
which has the same number of rows and columns as the 
(unpadded) image I(x.y). 

The reference and test images (optionally, following pre- 
processing) are converted from a grayscale format to local 
luminance contrast image. This conversion is depicted sche- 
matically as “Contrast” 101 in FIG. 1. The first step in this 
conversion or image transformation is the computation of a 
luminance image L(x,y) from the grayscales of each image, 
lest and reference, denoted generally as G(x,y) to indicate 
either G lesl (\,y) or G rrft , rence (\,y) respectively. For economy 
of language we frequently omit subscripts ”test” and “refer- 
ence” using a single unsubscripted letter to indicate two equa- 
tions or two variables, one each for “test” and “reference.” 

This transformation from grayscale G(x,y ) to a luminance 
image or luminance L(x,y) is advantageously performed by a 
gamma function “Gamma” as in Eq. 3. 

LU.v)=Gamma[GU.v)] Eq. 3 

The particular form and parameters used for the Gamma 
function will depend on the particular characteristics of the 
device displaying the test and reference images. A typical 
version is Eq. 4 in which the luminance L(x,y) is given by: 

Ux.y)-L mM iGix,y)IG max ) y Eq. 4 

in which L max is the maximum possible luminance in the 
image. G max is the corresponding maximum grayscale value, 
y is the gamma exponent of the display, approximately cor- 
recting for nonlinearities in the luminance characteristics of 
the display. A typical value for y is y=2.2. Eq.s 3 and 4 are 
applied to both test and reference images. 

A local luminance filter is employed having a luminance 
Filter function LF(x,y). It is then convenient to introduce a 
local mean luminance reference image LL(x,y) obtained by 
the convolution of the reference luminance image L reference 
(x,y) with the luminance filter function by Eq. 5a 

LUx,y)=LF(x.y)(^)L n!f€rmce {x,y) Eq. 5a 

in which (x) denotes convolution of the two functions defined 
in known texts in the field, for example “Fourier Analysis and 
Imaging” by Roger N. Bracewell, (Kluwer Academic/Ple- 
num Publishers, 2003), pp. 174-179, incorporated herein by 
reference. The convolution can be expressed in discrete and 
continuous forms as in Eq. 5b and 5c respectively. 

LF(x,y)(x)L Kfrrrnce (x,y)= 

liU 7 (x-x,y-<a)L rr/ , rence (x,y)dxd(0 Eq. 5b 

where the integrals extend over the domain in which LFd.co) 
is not zero. In discrete form the convolution is given by Eq. 5c. 


LF(x, y) ® L reference (x. y) = Eq- 5c 

^ LF(Mod(A-y, n x ), Mod (y-k, n y ))Lr fferfnCf (j, k) 

k =0 /=() 


where Mod(a, b) is the remainder when a is divided by b. 
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In some embodiments of the present invention, it is conve- 
nient to use the luminance filler function LF(x.y) given by Eq. 
6. 


LFU, y) = LF(r) E 9 6 

= — T Exp( — tt( — ) ) 

Iscale ‘ ' ' l sc ale ' ' 

r = y {xp*)r + (yp v ) 2 


in which Iscale is a parameter to be chosen. If iscale— H-©o, this 
corresponds to an LL that is constant and equal to the average 
luminance over the image. 

The average (MEAN) luminance, L rnean is given by a 
numerical average of the luminance over all pixels in the x 
and y directions, Eq. 7. 


~ t ^ Ln fercmM* .' ) 


A typical value for L llle(W is 40 candelas per sq. meter (40 
cd/nr). 

The contrast or contrast image of each pixel. C(x,y) is then 
given by Eq. 8 applied to both test and reference luminance 
images L„, v ,(x,y) and L reference (x,y) respectively. 


For the particular embodiments described thus far. L max 
plays no apparent role since it appears as a multiplicative 
factor in both L (Eq. 4), and LL (through L reference (Eq.s 4 and 
5)) hence canceling from Eq. 8. (Under the typically reason- 
able presumption that both test and reference images have the 
same maximum possible luminances, L„„ iV .) However, it is 
convenient to retain L, max in the equations since it simplifies 
the application of the equations in other embodiments of the 
present invention in which b max and/or L mean may play a role 
in determininu parameters of the process. A typical value for 
L m ^isl00cd/m 2 . 

Following the construction of test and reference contrast 
functions via Eq. 8, both test and reference images are typi- 
cally passed through a Contrast Sensitivity Filter (CSF). 102 
in FIG. 1. While various embodiments of CSF are feasible and 
can be used in connection with the present invention, it is 
advantageous in connection with some embodiments of the 
present invention to work in the frequency domain following 
application of a Discrete Fourier Transform, DFT, and its 
inverse DFT“ 1 . In such cases, the filtering can be described by 
Eq. 9 as 

F(x.y)=DFr'[CSF(u.v)*DFT[C{x,y))) Eq. 9 

in which C(x,y ) is the contrast function of the image from Eq. 
8 and F(x,y) is the filtered image. 

The Discrete Fourier Transform and the Inverse Discrete 
Fourier Transform, DFT[J and DF"P'[], are conventional 
operations in the field of digital signal processing and 
described in many texts, for example, the text by Bracewell, 
supra at pp. 167-168. incorporated herein by reference. 
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CSF(u.v) is the discrete version of a Contrast Sensitivity 
Filler in the frequency domain, and u and v are horizontal and 
vertical frequency indices respectively in units of cycles/ 
width and cycles/height. 

The discrete, frequency domain version of the Contrast 
Sensitivity filter, CSF(u,v) is conveniently given by the prod- 
uct of a radial contrast sensitivity function. RCSF(u.v), and an 
oblique effect contrast sensitivity filter, OEF(u.v). as 
expressed in Eq. 10. 

CSF(u,\')=RCSF(u.v)OEF(u,v ) Eq. 10 

In some embodiments of the present invention it is conve- 
nient to choose a radial function RCSF having the form given 
in Eq. 11. 

RCSF (w. v) = RCSF (/) Eq. 11 



in which “sech" is the hyperbolic secant function, “gain", 
“loss", f () , f,, and p are parameters. Typical values for these 
parameters are as follows: 

<0=4.173 
f ,=1.362 
loss=0.8493 
gain=373. 1 
p=0.7786. 

In some embodiments of the present invention, it is conve- 
nient to choose an oblique filter, OEF having the form given 
in Eq. 12. 

OEF (u. v) = OEF (f, ti) Eq. 12 

l i f - corner 

Siir(2fl)if / > comer 
= 1 if / < corner 



in which “comer" and “slope" are parameters. Typical values 
for “comer" and “slope" are comer=3.481 and 
slope= 13.57 149. 

Following processing of both the test image and the refer- 
ence image by CSF, 102 , the resulting filtered images are 
subtracted pixel-by-pixel. 103 . The result is the difference 
image D(x,y) of Eq. 13. 

0(x,y)— F lesr (x,y)~F re j- erenct Xx,y) Eq. 13 

In some embodiments of the present invention, it is advan- 
tageous to create a mask image, M(x,y), from the filtered 
reference image F /r/ ^„ re (x,y). In such embodiments, the 
absolute value of the filtered reference image is raised to a 
power “a", convolved with a masking filter MF(x,y ), added to 
the constant 1 and the b'th root of the resulting expression is 
computed as in Eq. 14. 


5 

10 

15 

20 

25 

30 

35 

40 

45 

50 

55 

60 

65 


11 


US 7,783,1 30 B2 


M (a, v ) = 1 1 + MF (a. v) <S> I F refWeni ,lx. yW 


Eq. 14 


in which the convolution operator (x) indicates discrete con- 
volution. 

In some embodiments, it is advantageous to choose a=b=2 
in Eq. 14, resulting in a mask image M(x,y) given by Eq. 15. 

i V/(.v.y)=V 1 +MF(x.y){x)F tK(ctvncc 2 (x.y) Eq. 1 5 

Furthermore, it is advantageous in some embodiments of 
the present invention to choose the masking filter MF(x.y) to 
have the form of Eq. 16 


MF(x . y) = MF (r ) = mg am Exp(-/r| 


mscale 1 


Eq. 16 


in which “mgain” and “mscale” are parameters. Typical 
choices for mgain and mscale are mgain=0.2 and mscale=(). 1 . 

In some embodiments of the present invention, the differ- 
ence image D(x,y ) is divided by the masking image to yield a 
masked difference image Ml)(x,y) according to Eq. 17. 


MD (a, v) = 


D (.v. y) 
M (a. y) 


Eq. 17 


For those embodiments in which a mask image is not 
employed, the masked difference image is simply the differ- 
ence image. Also, when a mask image is not employed, the 
subtract operation 103 can optionally precede the CSF 102. 

At this “boost” stage, 106 in FIG. 1. the absolute value of 
the masked difference image is computed, raised to a power p, 
and convolved with a window function W(x,y). The result of 
these operations is a function that is then raised to the power 
1/p and multiplied by the factor (p A .p v ) 1/p to produce a Just 
Noticeable Difference Image JND(x,y ) as in Eq. 1 8. A typical 
value for P is p=2.408 


i 

JND (a, y) = (p x p v )V 1 W (a. y) <g> \MD (a. 


i 



Eq. 18 


In some embodiments, it is advantageous to use a window 
function W(x,y) as given by Eq. 19 
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The next stage in the process combines or “pools” the 
values of JND(x,y) of the pixels in the x and y directions to 
produce a single value of JND. It is convenient to use a 
Minkowski summation to effect this pooling with a parameter 
5 \\f as exponent, as given in Eq. 20. 


JND - (p x p v )* 


10 


£ £ I JNDix. y)\* 


Eq. 20 


The number, JND of Eq. 20 is the desired numerical value 
characterizing the Spatial Standard Observer. 

15 In some embodiments, it is advantageous to let v|/— >oo, in 
which case Eq. 20 reduces to Eq. 21 . 

JND=Max[JND(x,y)] Eq. 2 1 

In some embodiments of the present invention, it is advan- 
20 tageous to apply a non-linear transformation (for example, a 
power function) to the JND computed from either Eq. 20 or 
Eq. 21. Thus, whether or not a non-linear transformation is 
applied to JND, and whether or not border effects are relevant 
for the particular image(s) under consideration, the Spatial 
25 Standard Observer, as characterized by the value of JND. 
provides an effective visibility metric, able to be computed 
relatively rapidly. 

In some applications, the target or test image (201 in FIG. 
2 ) may be located adjacent to a border 200 of the image region 
30 202. as depicted in FIG. 2. If the region of the display outside 
the image, 200. is darker than the display, 202. for example, 
the dark region of a Liquid Crystal Display (LCD) panel, then 
the visibility of the target 201 in the region will typically be 
reduced. An example of this situation is depicted in FIG. 2. 
35 Thus, it is advantageous in some embodiments of the present 
invention to use special techniques for the treatment of border 
areas in order to produce correct visibility estimates for such 
targets. 

In some embodiments of the present invention it is advan- 
40 tageous to multiply the contrast images by a spatial border 
aperture function BA(x,y) between the Contrast and CSF 
steps, that is, at 120 in the process How diagram of FIG. 1 . The 
resulting Contrast Border Aperture Function, CBA(x.y) is 
thus 
45 

CBA (a.v)=C(a.v )BA ( a.v) Eq. 22 

Then CBA(x,y) is used in place of C(x,y) at the CSF step, 
Eq. 9. 

In some embodiments of the present invention, the border 
50 aperture function is advantageously chosen to be: 


W (A, y) = W ( r ) = Exp l-n ( — ) ) 
' ' wscale ' ' 


in which “wscale” is a parameter, advantageously chosen to 
be approximately 1.013 in some embodiments. 

It is advantageous in some embodiments of the present 
invention to display the complete JND(x,y) image. 107 in 
FIG. 1. While optional, such a display can provide a useful 
visual indication of the location and magnitude of visual 
signals. The JND(x,y) image can be “thresholded” (setting to 
zero values less than a threshold, T), reduced in size (or 
otherwise scaled), and/or converted to a portable format to 
provide a compact record of the information. 


(a- \)p x , (y- \)p y , l- 
(n x - a )p x , (n y - y)p y j 
bscale 2 * 

60 in which “bgain” and “bscale” are parameters. An example of 
this function is given in FIG. 3 in which the image is taken to 
have 240 columns (x-coordinate) and 180 rows (y -coordi- 
nate). Each pixel in this example is taken to be ( l /6o) degree in 
both height and width. The parameters are chosen in this 
65 example as bscale=0.5 degree and bgain=l. 

The use of a border aperture function. BA(x,y) as in Eq. 23, 
has the advantage of simplicity, but as an approximation, it 


55 


BA (a, v) = 1 - bgain Exp -n 
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may not he as accurate as alternative methods. In other 
embodiments, it is advantageous for the parameters bscale 
and bgain to depend upon the luminance contrast between the 
image and the border. Typically, a margin is added to the 
image sue h that the enlarged image , image-f margin, contains 
a portion of the border. This enlarged image is then processed 
as the “image” pursuant to the image processing techniques 
described herein, typically including the masking component 
of the processing, 105. The presence of a portion of the border 
in the enlarged image will tend to produce the appropriate 
masking effect, tending to reduce visibility of targets or por- 
tions of targets near the border. 

There are various ways the use of an enlarged image can be 
implemented to treat border effects. For example, it is conve- 
nient to take the width of the border region to be Round 
[2*mscale/p v ] and the height to be Roundl2*mscale/p v |. in 
which mscale is the masking parameter (Eq. 16). “Round!]” 
is a function that generates as the value of the function that 
integer nearest to the value of the function's argument. The 
dimensions of the enlarged image are then given by Eq. 24 as: 

width=/j ,+Roundl 2*mscale//> J 

height=/i v +Round[2*mscale//> v ] Eq. 24 

An advantage of treating border effects with an enlarged 
image is that it more correctly deals with the dependence of 
the border masking effect upon the luminance contrast 
between the border and the (original, unenlarged) image. A 
possible disadvantage is that this approach requires some- 
what more processing to include the masking step. 

JND from Eq. 20 (or Eq. 21 for vjz-^oo) relates to the 
percentage of human observers who will notice a difference. 
For example, images leading to JND having a value around 1 
will typically present noticeable differences to about 75% of 
typical human observers. Images resulting in larger JND val- 
ues will present noticeable difference to a correspondingly 
larger percentage of typical human observers, although the 
precise functional relationship between JND and the percent- 
age of viewers observing differences may not be readily 
known. 

It is advantageous in some embodiments of the present 
invention to use JND as a measure of different levels of 
perceptual intensity. That is, larger JND values indicate that a 
larger percentage of observers will notice a difference. But 
also larger values of JND typically indicate that a given 
observer will be more likely to observe more detailed differ- 
ences. By way of illustration and not limitation, we consider 
the example of observing a scene through some form of 
optical instrument, such as a remote viewing device, night 
vision goggles, among others. A given observer may require 
an image value of JND, in order to conclude that some object 
is present other than natural background. However a value of 
JND 2 >JND, would be required for the observer to conclude 
that the object is a military vehicle. And a value of 
JND 3 >JND 2 would be required to conclude that it is a hostile 
military vehicle. Thus, JND values as determined by the SSO 
can be a useful measure of not only minimal levels of visibil- 
ity but, when more stringently applied, also estimate the 
probable level of perceptual information obtainable from a 
given image. 

FIG. 4 depicts an illustrative computer system 250 that 
utilizes the teachings of the present invention. The computer 
system 250 comprises a processor 252, a display 254, input 
interfaces 256. communications interface 258. memory 260, 
and output interfaces 262, all conventionally coupled by one 
or more busses 264. The input interfaces 256 comprise a 
keyboard 266, mouse, trackball or similar device 268. as well 
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as mass-storage input devices such as CDs, DVDs, magnetic 
discs of various designs among others. The output interface 
262 is a printer 272. The communications interface 258 is a 
network interface card (NIC) that allows the computer 250 to 
communicate via a network, such as the Internet. Image 
acquisition/generation devices 274 provide the images 100 
for the generation of the SSO and are also coupled to the 
processor 252. The units 274 can supply either stored or 
realtime input data, or both. 

The memory 260 typically comprises different modalities, 
illustratively semiconductor memory, such as random access 
memory (RAM), and disk drives. Depending on the embodi- 
ment. the memory 260 typically includes an operating sys- 
tem. 280. The operating system 280 may be implemented by 
any conventional operating system such as UNIX®. WIN- 
DOWS®, and LINUX®, among others. 

Although various embodiments which incorporate the 
teachings of the present invention have been shown and 
described in detail herein, those skilled in the art can readily 
devise many other varied embodiments that still incorporate 
these teachings. 

What is claimed is: 

1. A method of reducing wrap-around in a digital image 
processing process that compares a digital test image and a 
digital reference image and that includes one or more convo- 
lution steps, the method comprising: 

a) producing a reference luminance image from the refer- 
ence image and a test luminance image from the test 
image; 

b) producing a local mean luminance reference image as a 
convolution of the reference luminance image and a 
luminance filter function; 

c) producing a test contrast image that is at least one of the 
following: (cl) a mathematical combination of the test 
luminance image and the local mean luminance refer- 
ence image and (c2) a mathematical combination of the 
test luminance image, the local mean luminance refer- 
ence image and a border aperture function and (c3) a 
mathematical combination of the test luminance image, 
the local mean luminance reference image and an image 
of a border surrounding the reference image; 

d ) producing a reference contrast image that is at least one 
of the following: (dl ) a mathematical combination of the 
reference luminance image and the local mean lumi- 
nance reference image and (d2) a mathematical combi- 
nation of the reference luminance image, the local mean 
luminance reference image and the border aperture 
function; 

e) applying a contrast sensitivity filter to the test contrast 
image to produce a filtered test image; 

f) applying the contrast sensitivity filter to the reference 
contrast image to produce a filtered reference image; 

g) providing a difference image by at least one of the 
following two processes: (gl) subtracting the filtered 
reference image from the filtered test image to produce a 
difference image, and (g2) producing a mask image as a 
mathematical combination of the filtered reference 
image with a masking filter, and producing a difference 
image as a ratio of the difference image and the mask 
image; 

h) producing a just noticeable difference image as a math- 
ematical combination of the difference image with a 
window function; and 

i) pooling the just noticeable difference image to produce a 
visibility metric. 
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wherein the eonvolulion operation in at least one of process 
(b) and process (h) is performed using a confined con- 
volution process that comprises: 

(i-1 ) receiving an image I(x,y), expressed as an array of 
k*,,-by-k v<) pixels in an x-direction and in a y-direc- 
tion. respectively, where k x<) and k x<) are selected posi- 
tive integers: 

(i-2) padding the image I(x,y) with k A zeroes in the 
x-direction and by k v zeroes in the y-direction. to 
provide a first intermediate image Il(x.y). expressed 
as an array of (k rt) +k v )-by-(k ><) +k v ) pixels in the x-di- 
rection and in the y-direction, respectively, where k v 
and k v are selected non-negative integers: 

(i-3) convolving the first intermediate image 1 1 (x.y ) 
with a selected non-negative kernel function K(x,y), 
expressed as an array of k v -by-k v pixels in the x-di- 
rection and in the y-direction, respectively:, to obtain 
a second intermediate image 12(x,y) expressed as an 
array of (k rt ,+k v )-by-(k ><> +k v ) pixels in the x-direction 
and in the y-direction, respectively: and 
(i-4) cropping the second intermediate image I2( x.y ) to 
an array of k rt) -by-k >0 pixels in the x-direction and in 
the y-direction. respectively, to obtain a third interme- 
diate image I3(x,y), expressed as an array of k A<) -by- 
k <, pixels in the x-direction and in the y-direction. 
respectively. 

2. The method of claim 1, further comprising preprocess- 
ing at least one of said test image and said reference image by 
downsampling. 

3. The method of claim 1. further comprising preprocess- 
ing at least one of said test image and said reference image by 
convolution with a selected pre-filtering function. 

4. The method of claim 1, further comprising preprocess- 
ing, by cropping, at least one of said test image and said 
reference image. 

5. A method of forming a digital reference image from a 
digital test image, the method comprising: 

a) providing a test image having first and second opposing 
sides of the image 

b) performing a confined convolution of the test image with 
a selected filter that isolates the first and second oppos- 
ing sides of the image from each other, to thereby form 
the digital reference image. 
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wherein the confined convolution process comprises an 
ordered sequence of operations, denoted PCC, on an 
image, the sequence comprising: 

(i- 1 ) receiving an image I(x,y), expressed as an array of 
k Al ,-by-k v0 pixels in an x-direction and in a y-direc- 
tion, respectively, where k xi) and k A<) are selected posi- 
tive integers: 

(i-2) padding the image l(x.y) with k v zeroes in the 
x-direction and by k v zeroes in the y-direction, to 
provide a first intermediate image Il(x.y), expressed 
as an array of (k A<) -fk A )-by-(k >0 +k :v ,) pixels in the x-di- 
rection and in the y-direction, respectively, where k A 
and k v are selected non-negative integers: 

(i-3) convolving the first intermediate image 11 (x,y)with 
a selected non-negative kernel function K(x.y), 
expressed as an array of k A -by-k v pixels in the x-di- 
rection and in the y-direction, respectively:, to obtain 
a second intermediate image 12(x,y) expressed as an 
array of (k v0 -fk v )-by-(k A<) +k v ) pixels in the x-direction 
and in the y-direction, respectively: and 
(i-4) cropping the second intermediate image I2(x,y) to 
an array of k^,-by-k x<) pixels in the x-direction and in 
the y-direction, respectively, to obtain a third interme- 
diate image 13(x,y)=PCC {K(x,y), l(x,y)}, expressed 
as an array of k^-by-k^ pixels in the x-direction and 
in the y-direction, respectively. 

6. The method of claim 1, further comprising receiving said 
third intermediate image, I3(x,y)=PCC{K(x,y), I(x.y)} and 
forming a fourth intermediate image, defined as 

/4( a.v )=K(x.y)©l(x.y) 

=PCC{ K(x,y ). Kx.y ) }/PCC{ K( x.y )/I v I v K( x’.y’). 

I(x.y)}. 

7. The method of claim 5. further comprising receiving said 
third intermediate image. I3(x.y)=PCC{K(x,y). I(x,y)} and 
forming a fourth intermediate image, defined as 

/4< x.y )=K(x,y)©Hx.y) 

=PCC{K(x,y), I(x,y)}/PCC{K(x,y)/X t .I v .K(x , .y'), I(x, 

y)- 

* * * * * 
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